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ABSTRACT 

A possible method of solving problems of knowledge 
representation and system adaptation in a hypermedia environment was 
examined. The method, t>ased on tne notion of semantic networks, uses 
the technique of knowledge mapping. In the process of reading a 
hypermedia document , the learner constructs a knowledge map (KM) 
specifying the relationships between concepts in the domain* The 
resulting map can be used to recognize learner misconceptions and 
identify potential instructional opportunities • The prototype KM 
system was tested with 36 undergraduates assigned to 1 of 3 groups 
using different versions of the system (browse, map, and relate) , 
which allowed the learner to use a KM, construct a KM without 
feedback, or construct a KM with feedback. Subjects also completed 
paper and pencil pretests and posttests to identify the relationships 
between concepts introduced. All participants gained a significant 
amount of knowledge about the content. The browse group performed 
better on the short answer test, perhaps because participants in the 
other conditions were so involved in processing their KMs they did 
not study the concepts as well. A hypermedia system of this sort may 
provide an effective means of judging when a learner has attained 
some level of understanding of the domain. The system could then 
provide a new and appropriate level of detail for the learner. Four 
tables and five figures present study data* A 25-item list of 
references and 4 appendixes illustrating KMs and analyst procedures 
are included. (SLD) 
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Adaptive hypermedia instructional systems: Possibilities for learner modeling 

Much has been made of the parallels between the representational architecture of hypermedia 
and cuirent conceptions of human memory. Cognitive theories of learning emf^asize the 
interconnecticais of information in a structural associative nctwoiic composed of both the stored 
information and the relational links connecting the infonnation (Ancferson & Bower, 1973; 
Norman, Centner. & Stevens, 1976). The associative nature of these relaticHiships is also central 
to hypermedia systems, but few sy^ms allow authors to spedfy semantic relationships when 
defmmg linte between nodes (Collier, 1987). To facilitate the movement of hypennedia systems 
from sc^sticated information presentation systems towand effective lomwled^ representation 
systems for instruction, more attention should be placed on the underlying processes required for 
human knowledge acqmsiticm and representation in memory. Specific in^mry into the fundamental 
aspects of nodes (text fragments) and links fbranch^** to other nodes) is nece^ry in oixler for 
hypermedia systems to more readily parallel the ot^anization of human memory. In particular, it is 
important to examine various ways that hypermedia systems can convey to learners the underlying 
organization of the information in a domain. 

One of the major problems facing designers of hypermedia systems for education is to 
produce a system that does not simply present information, but fosters learning through effective 
pedagogical strategies (Nelson &. Palumbo, in press). Hypermedia provides ways to structure 
small, diiijrete units of information (nodes) into large documents which are viewed/read by 
following the paths (links) between nodes provided by tte author of the document The power of 
hypermedia applications lies in the ability to present infomiation in a nonlinear sequence, mirroring 
some of the assodational power of human memory (Collier, 1987). As a presentation medium, 
hypermedia offers the potential to provide large database of infonnation that can be traversed by 
leamere for a variety of purposes. But the motivational impact of leanwr-directed exploration of a 
hypermedia database is overshadowed by problems of disorientation and other difTicuIties (Duffy 
&. Knuth, 1991; Nelson & Joyner, 1990). Further, using hypermedia as a presentation system 
docs not guarantee that the information will be accurately and adequately transferred to the 
knowledge base of the learner (Nelson, in press). Establishing the most useful pathways through 
a hypermedia document with "tours" (Hammond, 1989) or "guides" (Orcn, Salomen, Kreitmen. & 
Don, 1990) can increase the likelihood that learners will view the critical infonnation, but there is 
little evidence that providing quick acce^ to a large body of information will promote learning any 
more than a library does. 

Few current hypermedia systems wrestle with the challenges of adaptive instruction as is 
common in other fomis of intelligent computer-assisted instruction. An adaptive hypermedia 
system that anticipates the needs of the learners, and dynamically adjusts the organization and 
presentation of infonnation accordingly (Oren, 1987), may be a more effective utilization of the 
technology for instruction. Aspects of artificial intelligence can be irKorporated in hypermedia 
systems to provide guidance to the learner in a flexible and efiicient manner (Barker, 1990). 
Adaptive access to large hypennedia databases could be achieved by considering the learner's prior 
history of interaction, current knowledge, and preferences. Decisions could then be made by the 
system with respect to which nodes and links to provide, how to organize the nodes, and which 
instructional activities to provide. 

The research reported in this paper examines a possible method to solve problems of 
knowledge representation and system adaptation in a hypermedia environment The method, based 
on the notion of semantic networks (Anderson & Bower, 1973; Collins & Quillian. 1969), uses 
the technique of knowledge mapping. In the process of reading a hypermedia document, the 
learner constructs a knowledge map specifying the relationships between concepts in the domain. 
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The resu|^g map can be used to recognize learner misconceptions, and identify potential 
instruction opportunities. 

The method: Eliciting knowledge structures 

Theories of a^ociative memory and semantic networks (Anderson & Bower, 1973; Collins 
& Quillian, 1969) assume that memory is organized as a netwcsic of concepts that are conna^ted 
through various relt^onships. It is possible to elicit an individual's conceptual organization using 
several techniques induding relatedness ratings (McKathcn, Reitman, Rcuter, & Hirtle, 1981; 
Schvaneveldt, Durson, Goldsmith, Breen & Cock, 1985), and pattern notes or knowledge maps 
(Jonassen, Such techniques can be used not only to elicit conceptual organization, but to 

convey infcmnation through spatial representations in order to facilitate the acquisition of 
declarative and prcKeduial infonnation, especially technical material (Rewey, Dansereau, Skaggs, 
Hall, & Pitre, 1989). Stuctents can also be taught to construct knowledge maps in order to sdidify 
understanding of the relationships between concepts in a domain (NovSc, 1990). It is apparent 
fnom this research that knowledge maps may provide the means for students to develop effective, 
personalized transitions between the infcmation in a hypermedia document and the cognitive 
structures of the learner 

The system described in this paper employs knowledge moping as a central learning activity, 
along with aspects of intelligent tutoring systems (Wenger, 1^7) that are used to analyze and 
identify student misconception. Intelligent tutoring systems typically utilize an "expert model" 
that rqjresents the knowledge of a dcsnain expert in sane format that can be stored in a computer 
(Anderson, 1988) and a "learner nKxfer that represents in a similar way the knowledge of the 
learner (VanLehn, 1988). An intelligent tutoring system can diagnose learner problems by 
examining the current state of the learner's knowledge in companson to the expert model (Orey & 
Nelson, in press), and then provide appropriate instructional interactions to help alleviate the 
problems. The system discussed below a^mbles an "expert" model as an expert defines concepts 
and relationships in- the domain, representing tbe relationships between concepts as a directed 
graph (Alty, 1984; Caire, 1979). The learner then uses the system to construct a knowledge map, 
and the system compares the learner's map (repTMented in the same format as the expert) to the 
expert model in order to identify and diagnose misconceptions. 

A prototype hypennedia system fitted with "tools" that allow leame.s to construct knowledge 
maps of a domain, using the resulting maps for diagnosis of learner misconceptions, was 
developed for this research. The system consists of two components: a Knowledge Editor and a 
Knowledge Map tool. The Knowledge Editor allows an "expert* (a teacher) to identify the 
concepts in the domain and to specify relationships between concepts. Information is entered by 
filling in forms (See Figure 1), and the concept, along with its definition and relationshii^s) to 
other concepts, is added to the database. The Knowledge Map tool aids the learner in constructing 
a map of the concepts and relationships in the domain. Initially, the map is blank, and the learner 
constructs the map by "placing" concepts in the two-dimensional space, and "connecting" concepts 
with lines to indicate the relationships (See Figure 2). At any time, the learner may view the 
definition of a concept by clicking on the concept name in the list, causing the concept description 
to be displayed in a window on the screen. In this way, the learner constructs a representation of 
knowledge in the domain which can be subsequently presented to others (or selH for discourse or 
study. 
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Figure L 

The Knowledge Editor. 
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Figure 2. 

The Knowledge Map tool. 
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There are several ways that an adaptive hyperm^a system such as the one d^cribed above 
could be used for instructional purposes. As one possifcsli^, the system could monitor the 
learner's activides as the map is being constructed For example, if a learner imiicates that an apple 
is a vegelal^e, Uie system would intervene at tbs point of die mistake and display further 
mformation <ksigned to modify the misconception. Alternatively, the mapfang activity could be 
used as a ''pretesr in orckr to detemiine the cument state of the learner's dcsnain Icnowl^ge. In 
this case tte learner would construct the map without intervention by the system. DiagiK)sis of 
misconceptions would occur after the learns had fmished the map. Bofi alternatives were 
examined in the r^earch reported below. 

Testing the prototype system 

Testing and evaluation of the Knowledge Map system focused on whether allowing learners 
to view and ccmstruct knowledge maps for a domain would facilitate acqui^tion and retention of 
the information, and whether the knowledge representation strategies employed in the system could 
be used to effectively diagnose and remediate learner misconceptions. In order to evaluate the 
system's performance, a comparative study was undertaken using three different versions of the 
software. 

Participants and Design 

Thirty six participants randomly selected frcsn a pool of undergraduates taking Educational 
Psychology classes used the Knowlwlge Map system during Fall, 1991 courses. The particioants 
were randomly assigned to one of three groups that utilized different versions of the Knov Jge 
Map system (Browse, Map, and Relate). A pretest-posttest design was employed in order to 
assess the knowledge gains of the participants when using the different versions o " the system. 

MateriaJs 

Three versions of the Knowledge Map system, ^h containing identical content, were 
developed for the experiment The systems were developed using HyperCard on the Macintosh 
asnputer. The content was adapted from a text on hypertext (Shneiderman & Kearsley, 1989), 
and was entered using the Knowledge Editor. One version (Browse) was developed to present the 
information using a foiowledge map onstnicted by an expert (See Appendix A). A ^cond version 
(Map) was developed so that particij»nts <x5uld use the Knowledge Map tool to construct a map of 
the domain (See Appendix B), but no feedback was provided to the participants using this version. 
The third version (Relate) was identical to the se<XMid, except that an expert model was included, 
and the software was programmed to provide instructional feedback in order to help learners 
construct maps that were identical to the expert Specifically, the Relate version informed the 
learners when an attempt to connect unrelated concepts was made, and refused to make 
connections that were not identified in the expert model. In addition, this version of the software 
also required participants to specify the type of relationship between concepts when making a 
<X)nnection (See Appendix C). All vereions of the software contain©! instrumentation that recorded 
the participant's interactions with the software, as well as the diagncses made by the system. 

In addition to the different software versions, paper-and-pengil tests were constructed for use 
in the pretest-posttest experimental design. These tests contained 13 multiple choice tests items that 
required the f«rticipants to identify the type of relationship between pairs of concepts. The tests 
also contained 13 short-answer items that required the participants to write a short definition of 
each <x)ncept in the domain. 



Learner modeling for hypermedia 

6 



Procedure 



Partiapants were pretested on their knowledge of the content me week prior to using the 
Knowledge Map system. To test the software, participants worked with the Knowledge Map 
system for two hours during cme of the regulariy scheduled riass periods. They were trainSi to 
use the system through demonstration and practice with a system that operated exactly like the 
prototype system, but contained different content Printed reference materials desaibing the 
qjcration of the software were used for the training, and wens available to the participants during 
thecxpenment When the participants were comfortable using the practice system, they proceeded 
with me expenmental task. AU participants were informed that they should complete the 
experimental t^k m preparation for a test on the content contained in the program. P&itidpants in 
tl« Browse goup accessed the information in tl» map by clicking otk the various map components 
^reading the information presented Participants in the other two groups (Map and Relate) used 
the Kmjwledge Map system to construct »aphical representations of the concepts and relationships 
m the domain. All participants completed the paper-and-pendl posttest immediately after fmishing 
the expenmental task A delayed measure that required participants to produce a map of the 
concepts m the domain was administered three weeks after the experimental session. 

Results and Discussion 

The dato collected in the study included the matrices maintained by the software that 
represented the Knowledge Maps constructed by each participant (Map and Relate groups only) 
me maj^ constructed by the participants (as a result of me experimental task and th^ weeks ' 
delayed), the pretests and posttests that assessed knowledge gained by using the software, and 
qualitative data a)llected by the software describing each participant's patterns of interaction 
Analysis focused on me diagnostic capabilities of me software used by me Relate group the 
COTceptual organization exhibited by participants who used me different software vereions, 
qualitative descnphons of me strategies participants developed when using me software and 
ccMipansons of learmng as evidenced by performance on me tests. 

Diagnostic Capabilities 

The diagnostic capabilities of me Knowledge Map system were evaluated in several ways 
pnmanly through comparisons between me results of me Map group and me Relate group, since 
the Browse group did not constmct knowledge maps. The software maintained a daS stmcture in 
me torm of a matnx that represented me maps constructed by the participants. These matrices were 
analyzed to determine the total number of links made between concepts, the number of correct 
links, and me number of incorrect links (See Table I). A link was classified as incorrect if 
companson to me expert model showed that a link was identified mat should not have been made 
or If a link was not identified when it should have been. An ANOVA perf-ormed on me data 
!?^^lr T ^'P^f'^^ differences between groups for total links identified (F[l,22] = 1 742 p < 
U.ZUU5), but mere were significant differences for correct links (HI, 221 = 102 414 d<Q OOOn 

!2?„n''I!f '^^r"^.^^^'^^^^ ^^^-^^^'^^^-OOOIX Scheffd's test confirmed th^ the Relate 
group made sigmficanUy more correct links, and significantly fewer incorrect links. 

This result is not surprising, since the software for me Relate group did not allow participants 
to Identity an incorrect link When a user attempted to connect two concepts mat were not 
specified as re at^ in the expert model, me software informed me user mat me two concepts were 

u 5 o diagnostic/instnictional process inherent in me sofuvare 

Zltli ,!l r^f P produced anomer interesting result me correct number of links and 
S t£ i"^ participants in me Relate group were me same. However, some links 

n the Relate group to were classified as incorrect in me analysis were actually missing because 
me participant ended me session wimout connecting all concepts in the map. In me aniysis 
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procedure, if no link was made when it should have been, the score for incorrect links was 
imrreased. In thas sense, "incorrect" means "not identified at air, not "incorrectly identified*'. 
Future versions of the software will have to assure that learners do not quit until all concepts are 
correctly connected in the map. 

Table 1. 



Mean number of linked concepts in Map and Relate conditions. 



Group 


Total Links 


Correct 


Incorrect 


Mean S. D. 


S. D. 


Mean S. D. 


Map (N = 12) 


13.25 3.646 


4.25 2.094 


9.75 2.094 


Relate (N= 12) 


11.75 1.485 


11.75 1.485 


2.25 L485 



The software used by the Relate group also required participants to identify the type of 
relationship between two concepts when connecting the concepts on the map. Results from the 
interaction data files were examined to determine the total number of attempt made to identify the 
relationship between concepts, and the number of connect and incorrect identifications of relation 
types. The means for these categories are presented in Table 2, alon^ with the the percentage of 
attempts that were incorrect Tte 46% figure fa* incorrect identificaticm of the type of relationships 
between concepts is quite high, and represents an interesting and unanticipated strategy that 
participants utilized. Observation of the participants during the experiment revealed that when 
promfrted by the software to identify the type of relationship between two concepts, the participants 
frequently just "guessed". The sd"tware Uien informed the user of the error, and the user was 
given another chance to identify the type of relationship. Participants typically just clicked on 
different relation types until the system let them continue because they had finally selected the 
correct relation. Subsequent versions of the software will have to be modified in order to 
discourage this strategy, but it was interesting to see the strategy develop spontaneously as the 
participants used the system. 

Table!. 



Numbers of correct and incorrect relaUons identified by participants in the Relate group. 





Total Attempts 


Incomect 


Correct 


% Incorrect 


Mean (N=12) 


25.76 


13.29 


12.47 


46.0 


S. D. 


8.34 


9.92 


3.02 


24.0 



Conceptual OrganizjOtion 

One of the major purposes of the Knowledge Map software is to help learners construct maps 
that reflect the organization of concepts in a domain as specified by an expert If the system is 
successful, there should be significant differences between the resulting conceptual organization of 
the Relate group (where the software forced the learners to ojnstruct maf^ that were identical to the 
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expert representation) and the Map and Browse groups (where the software did not force 
adherence to the expert model). As mentioned eariier, techniques for eliciting and analyzing 
coiK;eptual organization have been developed that can provide indications of the effectiveness of the 
software for meeting its goals. Since participaats in the Map and Rdate groups produced maps 
both as a part of the experimental task (immediate), and as a delayed measure thnec weeks 
following the experiment (delayed), data was available for comparisons between versions of the 
software. In addtion. since all three groups completed concept maps three weeks after the 
experiment, data existed to compare the conceptual organization of all partidpants after some delay. 

An initial assumption made for puiposes of analysis was that the maps constructed by the 
participants could be ti^ited as directed graphs (Jona^n. 19S7). The distance buhveen two 
concepts on the maps was defined as the number of lines between pairs d* ccn!cept« n the map. A 
distance matrix was formed for each participant by counting the intervening lines between each o€ 
tl^ 13 concepts (See Appendix D fcH- an example). The distance matrix for each participant was 
then compared with a matrix formed by analyzing the expert representation in the same way Each 
cell in the participants* matrices was then subtnK^ted from the corresponding cell in the expert 
matrix, and the results were used to construct a difference matrix for each participant The 
difference matrices for each participant in the Relate and Map groups were then compared using a t- 
test procedure between each cell in the difference matrices. Cells where significant differences 
were found by the t-test procedures arc given in Figure 3 for the immediate measures. 

Figure 3. 

Significant differences (p < 0.03) between Map and Relate groups for knowledge maps 
immediately after experimental task. 
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All of the significant differences identified by the t-test procedures as shown in Figure 3 were 
in favor of the Relate group. That is, in all cases the differences between the Relate group and the 
expert model were significantly less than th« differences between the Map group and the expert 
model- These results indicate that the conceptual organization of the participants in the Relate 
group was more similar to the expert model than the participants in the Map group immediately 
following tte experimental task. Again, this is not surprising since the software used by the Relate 
group did not allow users to connect concepts that were not COTinected in the expert model. 
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Similar analyses were completed on the maps constructed by participants after a three week 
dday. Bgure 4 shows the results of the t-test procedures betw^ the Browse and Map groups, 
the Browse and Relate groups, and the Map and Relate group. Some of the differences in 
conceptual organization woe maintained even during a three week delay. It is interesting to nc^, 
however, that the differences between the Map and Relate groups do not remain between die same 
concept pairs as in the immediate measures. Instead, new difTereiK^es between other concept pairs 
emer^ in the delayed measure. 

Rgure4. 

Significant differences between groups for knowledge maps after three week delay. 
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A second phase of analysis utilized nonmetric multi-dimensional sailing (MDS) procedures 
to map the semantic distances between concepts. For the delayed measure, all of the pairwise 
vdlvss in the matrices for each paiticipant were averaged to jxniuce a mean distance matrix for 
each pair of concepts in the three groups (See Appendix D for an example). It can be assumed that 
the higher the mean for a given pairwise value, greater the spatial distance between those 
concepts in the maps, and therefore the less tl^ two concepts are related (Jona^en, 1987). The 
mean distance matric^ were scaled using the statistical procedure ALSCAL (SPSS-X User's 
Guide, 1988). Goodness-of-flt measures were computed using Kniskal's Stress Formula I for 
each group*s avenge distance matrix, with an RSQ \^ue indicating the proportion of variance 
accounted for by the MDS model. For the Browse model, stress = 0.2SS, RSQ = 0.682, for the 
Map model, stress = 0302, RSQ = 0.458, and for the Relate model, stress = 0.29, RSQ = 0.51 1. 
Since stress values of 0. 10 are considered good (Jonassen, 1987), the stress and variance of the 
models in this experiment are relatively high. However, the str^ values are not the critical issue 
for purposes of this smdy. Rather, the plotting of the stimulus weights as computed by the MDS 
procure (Figure 5) provides the information of interest, that is, the spatial maps of the average 
distances between concepts for each group. 

Figure 5. 

MDS solutions for average concept organization of each group. 
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Some interesting consistencies between groups are evident in the MDS solutions plotted in 
Figure 5. Brst, the "hypertext'* concept does not cluster with any other concepts, perhaps 
indicating a superordinate relationship, with other concepts clustered around it. The same is true of 
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the "structure" concept and its relationships to "networks" and "hierarchies**. Other concepts tend 
to cluster in all MDS solutions, such as **networks" with "hierarchies", and "browsing** with 
**authoring'*. It is also interesting to note that the concepts in the MDS solution for the Browse 
group tend to cluster atx3und the center, while the Map and Relate group appear to have more 
"spread** between concepts. 

Qualitative Results 

The software used by participants in the Map and Relate groups recorded every action taken 
by each participant while using the software. The data from these nies, coupled wiik observations 
made by the experimenters, give an indication of the differences in the ways that participants 
approached the experimental task. The mean frequences of each type of action are summarized in 
Table 3, includmg results of ANOVA pnx^ures comparing the means between Map and Relate 
groups. As can be seen, there were no significant difference between groups except for the 
"Remove** action. Participants in the Map group removed more conce|^ from the map than did 
participants in the Relate group. Although the reasons for this difference are not clear, the fact that 
some concepts were removed from the map may have omtributed to the differences in conceptual 
organization that were noted earlier. Observations by experimenters indicate that it is also possible 
that the concepts were removed for "cosmetic** reasons, that is, participants were trying to "clean 
up*' the appearance of their maps, and eventually placxd the concepts back on the maps. 

Table 3. 



Means and ANOVA results of various actions taken by participants while using the soflv j:e. 





Map Group 


Relate Group 








Mean S. D. 


Mean 


D. 


F value 


P 


View 


36.75 27.13 


31.08 


7.9 


.483 


.494 


Place 


12.25 7.45 


17 


5.33 


3.23 


.086 


Move 


95.08 55.41 


105.4 


39.5 


.277 


.604 


Connect 


23.67 16.26 


35.42 


19.66 


2.54 


.125 


Disconnect 


4. 17 9.97 


.08 


.29 


2.011 


.1702 


Remove 


2 1.86 


.67 


1.07 


4.632 


.043 


Redraw 




6.25 


6.97 


3.67 


.094 



The qualitative data collected by the software also indicated difference in strategies used by 
participants to complete the moping activity. Some [^rticipants tended to flrst place all of the 
concepts in the map, then link the concepts and rearrange the map in order to make the map easier 
to read Other participants tended to place and link concepts in pairs, keeping the map in acceptable 
spatial organization as they placed each pair of concepts. Finally, some [^rticipants placed and 
linked the concepts using one or the other the strategies mention^ then viewed all of the 
concepts repeatedly. Apparently, they were studying the map in preparation for the posttest These 
different strategies appeared to develop spontaneously, siiK^ the experimenters did not advocate a 
certain strategy for the task. It should be noted that these strategies were used by participants in 
both groujK, indicating that the difference in the software did not influence the strategies used by 
participants to comfrfete the task. 

Pbst-experiment comments made by some participants also indicated improvements that 
could be made to the Knowledge Map software. First, several participants wanted to the map to 
specify the type of relationship between concepts, rather than just a line indicating that the concepts 
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are related Some f^rcicipants in the Relate group also express^ frustraticm that the "^expert 
moder did not allow them to ccsistnict relationships that they felt existed. Apparently they wanted 
to construct a richer network in terms of relationships betweeti concepts, but the expert model was 
not designed to allow such richn^. 

Measures o/Learmng 

i 

Traditional measures of learning utilizing paper and pencil tests were also used to evaluate the 
efTectiveQess of the Knowled^ Map system. As mentioned earlier* multiple choice and short 
answer test items were administered to all paiticipants as a pret^ and as a postt«:st immediately 
following the experim^tal task. Gain scores were computed for all participants by subtracting the 
pretest scores from the posttest scores. The mean gain scc^ for each group are ^ven in Table 4. 
ANOVA procedures revealed no stgnificant differences b^wecn groups on the multiple choice 
items (F[233] = 0.81 l,p< 0.453), but there were significant differences on the short-answer 
items (F[233] = 3.341, p < 0.0477). A Scheff^ test indicated that the gain scores for the Browse 
group were sigmficantly greater than the Map group, but not the Relate group. 

Table 4. 



Mean gain scores on paper-and-pencil tests. 



Group 


Multiple Choice 


Short Answer 




Mean S. D. 


Mean S. D. 


Browse (N = 12) 


3.0 2.256 


7.583 3.397 


Map(N= 12) 


1.197 2.999 


3.917 3.872 


Relate (N = 12) 


1.583 3.204 


5.417 0.917 



All participants, regardless of experimental group, gained a significant amount of knowledge 
about the content Psdred t-tests showed significant differences between the pre and post multiple 
choice items for the Browse group (/ = 4.606, df= II, p <.0008), the Map group (t = 2.214, df 
= U, p <.0489), but the Relate Group did not show significant improvement on the posttest for 
multiple choice test items (/ = 1.715, df - 1 1, /? < 1 149). Similar improvements between the pre 
and post short-answer items were revealed for the Browse group (/ = 7.734, d/= II, p <.O001 ), 
the Map group (t = 3.504. df~ 1 1, p <0O49), and the Relate Graup (t = 5.909, df=\\,p 
<.00OI). The fact that the Browse ^up performed better on the short-answer test was not 
expected, but can be easily explained. The cognitive load associated with the Knowledge Map 
software for the Map and Relate versions must have interfered with the processing of the content 
Participants were so involved with constructing their maps that they did not take the time to read 
and study the descriptions of the concepts. They were only interested in determining the 
iielationships between concepts that were indicated in the concept descriptions so that they could 
connect the appropriate ccMicepts in the map. 

Conclusions: Future directions 

Further testing of tiie ^proach to learner modeling implemented in the Knowledge Map 
software is needed, but if this or other methods of knowledge representation and diagnosis prove 
valid and useful, futiire hypermedia systems could use the knowledge maps constiiicted by a 
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learner to organize and Tiller" information in a hypennedia documenL Using the Knowledge Map 
toot in a "[»etest** mode, a learner could specify the or^uiization of a large document, and the 
system could then structure tbs informatioQ to ad^t to the learner^ uodorstanding of the domain. 
At the same time, the syst^n could identify misconceptions and recommend furth^ study of the 
areas where problems were identified Such an approach may reduce the disorientation often 
experienced by learners when browsing a hypennedia document, and may better facilitate learner 
exploration and ulttmate transfer of the infomiation in the hypermedia system to the ieamer*s 
memory. 

Further re^arch also needs to focus on when is the best time for interv^tion by the system. 
The method imf^emented with the Relate group provided immediate feedback whm an error was 
made. As noted above, participants who used the Relate versicm of the software expr^sed 
frustration that the system was so inflexible, and many participants adc^ted a "'guessing'* strategy 
to identify the type <^ rdationship between ccmcepts, rather Umn reading the concept d^riptions 
and inferring the type of relatioi^p through reasoning processes. On the other hand, using this 
version of the software resulted in conceptiml organization that was generally more similar to the 
"expert** than the other versions. 

The richness of the expert model also pc^ed {HX>Hems in this study. Participant in the Relate 
group mentioned that they felt there were relationships betw^n ccmcepts that the system would not 
let them identify; that there were more relationships tl^ should have been allowed. This problem 
could be alleviated by adopting a representation that uses weightings instead of an ^all or nothing** 
representation of expert The expert mode! implemented in this study m^y indicated that a pair 
of concepts was relate, not the degree to whidi all concepts were related to all other concepts. 
Future versions of the software will need to address this problem. 

Finally, a hypennedia system that contains an expert model and a method for learner 
modeling such as the mapping activity used in the Knowledge Map software may provide an 
effective means of judging when a learner has attained some level of understanding of the domain. 
The system could then provide a new level oi <ktsdl that is appropriate to the learner's current 
knowit^ge. In order to imj^ement such a hypermedia system, it wculd be necessary to organize 
the hypermedia database in "layers" of increasing specificity so that learners could be presented 
more detailed descriptions and relationships after they had mastered the initial content This would 
require that maps be three-dimensional, increasing the complexity of the learner's task as well as 
the methods used for diagnosis and knowledge repr^ntation. Further research is needed to 
determine if the benefits of such a system are worth the effort 
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Appendix A 

'Expert** map used by the Browse group 



ikuthol 




Compgttr instruction 



hyptrttxt 



I 



structirt 




hltr^-chits 





tourf 



C Quit 



■s Click on « box to Mtt « (i^fcr^rtion of tht oonotpt. Click cn tht ^^rows it tht 

-/ fi<l# of th# win4ow to scroll up *nd dcvn. Click on th* vindov to st# th» nwp again. 
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Appendix B 

Knowledge Map system used by the Map group 



Knoffied^liiftp 




Help )^^— : — Pt«J« 

C Disconnect jC Connect )C RePrav j 



Remove 



C e» otp t List 

CMii(»it«r fnstruotf on 

links 

nodm 

hypertext 

tours 

wtworks 

hitr^'chfts 
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Appendix C 

Knowledge Map system used by the Relate group 



How are Computer instruction end hypertext related? 

Click on the relation ty^ below: 
ia an example of 
i5 a proi»rty of 
ie identical to 
la similar to 
18 not similar to 
is greater than 
occurs before 
caus^ 
allows 



VMw^K List 




tours 
iwtvorks 
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Appendix D 

Analysis procedures 



The Expert's Knowledge Map 



Aquatic Birds 



igbUess Birds 




Auks 



Birds 




The distance matrix derived from the map 

I I 
li 



Ostridi 



il < If 1 1»| 



Bircb 0 
Feadim 1 0 
Aquatic Btids 1 2 0 
Auks 2 3 10 
FlfghUess Birds 1 2 2 3 0 
Ostrich 2 3 3 4 1 0 



The Student's Knowledge Map 



Aqt»tic Btfds 



Birds 



Flightless Birds 



The distance matrix derived from the map 

25 







Ostridt 






Fcathm 



j 



Auks 



niS 44 

Birds 0 

Feathers2 0 

Aquatic Binb 12 0 

Auks 13 2 0 

Fli^tfess Birds 1 3 2 2 0 

Ostrich 1 1 2 2 2 0 



For each cell in the matrix 

The Student Model . . . minus the Expert Model . . . 



M 

Birds 0 
Feathers 1 0 
Aquatic Birds 0 0 0 
Auks 0 0 10 
Flightless Birds 1 0 0 0 0 
Ostrich 1 0 0 0 1 0 



I I II 1-1 



Birds 0 
Feathers 2 0 

Aquatic Birds 12 0 

Auksl 3 2 

Rightiess Birds 1 3 2 

Ostrich 1 I 2 



0 

2 0 
2 2 0 



equals the difference matrix 

IS » 

Birds 0 
Feathers -1 0 
Aquatic Birds -1-2 0 
Auks -1-3 1 0 
Flightless Birds 0-3-2-2 0 

Ostrich 0-1 -2-2-1 0 



The average value for each 
ceil of each group was 
used for the 
MDS procedure. 



Each cell of each 
participant's difference 
matrix was compared 
using t tests. 



